Goto

Collaborating Authors

 training material


"Think First, Verify Always": Training Humans to Face AI Risks

Aydin, Yuksel

arXiv.org Artificial Intelligence

Artificial intelligence enables unprecedented attacks on human cognition, yet cybersecurity remains predominantly device-centric. This paper introduces the "Think First, Verify Always" (TFVA) protocol, which repositions humans as 'Firewall Zero', the first line of defense against AI-enabled threats. The protocol is grounded in five operational principles: Awareness, Integrity, Judgment, Ethical Responsibility, and Transparency (AIJET). A randomized controlled trial (n=151) demonstrated that a minimal 3-minute intervention produced statistically significant improvements in cognitive security task performance, with participants showing an absolute +7.87% gains compared to controls. These results suggest that brief, principles-based training can rapidly enhance human resilience against AI-driven cognitive manipulation. We recommend that GenAI platforms embed "Think First, Verify Always" as a standard prompt, replacing passive warnings with actionable protocols to enhance trustworthy and ethical AI use. By bridging the gap between technical cybersecurity and human factors, the TFVA protocol establishes human-empowered security as a vital component of trustworthy AI systems.


The US Army Is Using 'CamoGPT' to Purge DEI From Training Materials

WIRED

The United States Army is employing a prototype generative artificial intelligence tool to identify references to diversity, equity, inclusion, and accessibility (DEIA) for removal from training materials in line with a recent executive order from President Donald Trump. Officials at the Army's Training and Doctrine Command (TRADOC)--the major command responsible for training soldiers, developing leaders, and shaping the service's guidelines, strategies, and concepts--are currently using the AI tool, dubbed CamoGPT, to "review policies, programs, publications, and initiatives for DEIA and report findings," according to an internal memo reviewed by WIRED. The memo followed Trump's signing of a January 27 executive order entitled, "Restoring America's Fighting Force," which directed Defense Secretary Pete Hegseth to eliminate all Pentagon policies seen as promoting what that the commander-in-chief declared "un-American, divisive, discriminatory, radical, extremist, and irrational theories" regarding race and gender, a linguistic dragnet that extends as far as past social media posts from official US military accounts. Chris Robinson confirmed the use of CamoGPT to review DEIA materials. "[TRADOC] will fully execute and implement all directives outlined in the Executive Orders issued by the President. We ensure that these directives are carried out with the utmost professionalism, efficiency, and in alignment with national security objectives," Robinson says.


Are you 80% angry and 2% sad? Why 'emotional AI' is fraught with problems

The Guardian

It's Wednesday evening and I'm at my kitchen table, scowling into my laptop as I pour all the bile I can muster into three little words: "I love you." My neighbours might assume I'm engaged in a melodramatic call to an ex-partner, or perhaps some kind of acting exercise, but I'm actually testing the limits of a new demo from Hume, a Manhattan-based startup that claims to have developed "the world's first voice AI with emotional intelligence". "We train a large language model that also understands your tone of voice," says Hume's CEO and chief scientist Alan Cowen. "What that enables… is to be able to predict how a given speech utterance or sentence will evoke patterns of emotion." In other words, Hume claims to recognise the emotion in our voices (and in another, non-public version, facial expressions) and respond empathically.


Generative AI Models: Opportunities and Risks for Industry and Authorities

Alt, Tobias, Ibisch, Andrea, Meiser, Clemens, Wilhelm, Anna, Zimmer, Raphael, Berghoff, Christian, Droste, Christoph, Karschau, Jens, Laus, Friederike, Plaga, Rainer, Plesch, Carola, Sennewald, Britta, Thaeren, Thomas, Unverricht, Kristina, Waurick, Steffen

arXiv.org Artificial Intelligence

Generative AI models are capable of performing a wide range of tasks that traditionally require creativity and human understanding. They learn patterns from existing data during training and can subsequently generate new content such as texts, images, and music that follow these patterns. Due to their versatility and generally high-quality results, they, on the one hand, represent an opportunity for digitalization. On the other hand, the use of generative AI models introduces novel IT security risks that need to be considered for a comprehensive analysis of the threat landscape in relation to IT security. In response to this risk potential, companies or authorities using them should conduct an individual risk analysis before integrating generative AI into their workflows. The same applies to developers and operators, as many risks in the context of generative AI have to be taken into account at the time of development or can only be influenced by the operating company. Based on this, existing security measures can be adjusted, and additional measures can be taken.


OpenAI partners with People publisher Dotdash Meredith

Engadget

OpenAI is partnering with another publisher as it moves towards a licensed approach to training materials. Dotdash Meredith, the owner of brands like People and Better Homes & Gardens, will license its content for OpenAI to train ChatGPT while the publisher will use the AI company's models to boost its in-house ad-targeting tool. As part of the arrangement, ChatGPT will display content and links attributed to Dotdash Meredith's publications. It also provides OpenAI with fully licensed training material from trusted publications. That's a welcome change after the company got in hot water for allegedly using content for training purposes without permission.


Establishing Vocabulary Tests as a Benchmark for Evaluating Large Language Models

Martínez, Gonzalo, Conde, Javier, Merino-Gómez, Elena, Bermúdez-Margaretto, Beatriz, Hernández, José Alberto, Reviriego, Pedro, Brysbaert, Marc

arXiv.org Artificial Intelligence

Vocabulary tests, once a cornerstone of language modeling evaluation, have been largely overlooked in the current landscape of Large Language Models (LLMs) like Llama, Mistral, and GPT. While most LLM evaluation benchmarks focus on specific tasks or domain-specific knowledge, they often neglect the fundamental linguistic aspects of language understanding and production. In this paper, we advocate for the revival of vocabulary tests as a valuable tool for assessing LLM performance. We evaluate seven LLMs using two vocabulary test formats across two languages and uncover surprising gaps in their lexical knowledge. These findings shed light on the intricacies of LLM word representations, their learning mechanisms, and performance variations across models and languages. Moreover, the ability to automatically generate and perform vocabulary tests offers new opportunities to expand the approach and provide a more complete picture of LLMs' language skills.


Do LLMs Dream of Ontologies?

Bombieri, Marco, Fiorini, Paolo, Ponzetto, Simone Paolo, Rospocher, Marco

arXiv.org Artificial Intelligence

Large language models (LLMs) have recently revolutionized automated text understanding and generation. The performance of these models relies on the high number of parameters of the underlying neural architectures, which allows LLMs to memorize part of the vast quantity of data seen during the training. This paper investigates whether and to what extent general-purpose pre-trained LLMs have memorized information from known ontologies. Our results show that LLMs partially know ontologies: they can, and do indeed, memorize concepts from ontologies mentioned in the text, but the level of memorization of their concepts seems to vary proportionally to their popularity on the Web, the primary source of their training material. We additionally propose new metrics to estimate the degree of memorization of ontological information in LLMs by measuring the consistency of the output produced across different prompt repetitions, query languages, and degrees of determinism.


Synthetic Data Is a Dangerous Teacher

WIRED

In April 2022, when Dall-E, a text-to-image visio-linguistic model, was released, it purportedly attracted over a million users within the first three months. This was followed by ChatGPT, in January 2023, which apparently reached 100 million monthly active users just two months after launch. Both mark notable moments in the development of generative AI, which in turn has brought forth an explosion of AI-generated content into the web. The bad news is that, in 2024, this means we will also see an explosion of fabricated, nonsensical information, mis- and disinformation, and the exacerbation of social negative stereotypes encoded in these AI models. The AI revolution wasn't spurred by any recent theoretical breakthrough--indeed, most of the foundational work underlying artificial neural networks has been around for decades--but by the "availability" of massive data sets.


CDC linked to pervasive curriculum sweeping public schools nationwide

FOX News

Dukes and Jackson, both with No Left Turn in Education, said parents should be concerned about how AI is being used in schools, and what information it may gather on students. Educators at over 120 districts across the country are implementing a pervasive school curriculum that has been denounced by opponents as an effort to manipulate children's values and beliefs and replace parents as the primary moral authority in their child's lives, with many critics specifically pointing to similarities with programs from the Centers for Disease Control and Prevention (CDC) as a major point of contention. The School Superintendent's Association (AASA), with the help of superintendents, board members and school administrators, is implementing the Learning 2025 program, which calls for an equity-focused, "holistic redesign" of the United States' public education system by 2025, in districts across the country The parents' advocacy group, No Left Turn in Education (NLTE), is sounding the alarm about the curriculum's alleged ties to the CDC, especially since Learning 2025 outlines its plans as a solution to the fallout of the COVID-19 pandemic. Learning 2025 frequently references the idea of a "Whole Child" educational framework to promote the notion that school districts should focus on a collective, whole community vision that is strikingly similar to the Whole School, Whole Community, Whole Child (WSCC) educational framework devised by the CDC. Both programs place a strong emphasis on students' and teachers' social and emotional health, including employee wellness programs, as well as psychological and social services like school-based health and counseling centers.


Fresh concerns raised over sources of training material for AI systems

The Guardian

Fresh fears have been raised about the training material used for some of the largest and most powerful artificial intelligence models, after several investigations exposed the fascist, pirated and malicious sources from which the data is harvested. One such dataset is the Colossal Clean Crawled Corpus, or C4, assembled by Google from more than 15m websites and used to train both the search engine's LaMDA AI as well as Meta's GPT competitor, LLaMA. The dataset is public, but its scale has made it difficult to examine the contents: it is supposedly a "clean" version of a more expansive dataset, Common Crawl, with "noisy" content, offensive language and racist slurs removed from the material. But an investigation by the Washington Post reveals that C4's "cleanliness" is only skin deep. While it draws on websites such as the Guardian – which makes up 0.05% of the entire dataset - and Wikipedia, as well as large databases such as Google Patents and the scientific journal hub PLOS, it also contains less reputable sites. The white nationalist site VDARE is in the database, one of the 1,000 largest sites, as is the far-right news site Breitbart.